Skip to content

MAINT: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback#2053

Open
rlundeen2 wants to merge 22 commits into
microsoft:mainfrom
rlundeen2:rlundeen2/unify-red-team-attack-schema
Open

MAINT: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback#2053
rlundeen2 wants to merge 22 commits into
microsoft:mainfrom
rlundeen2:rlundeen2/unify-red-team-attack-schema

Conversation

@rlundeen2

@rlundeen2 rlundeen2 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

AdversarialConversationManager

AdversarialConversationManager centralizes how multi-turn attacks talk to their adversarial chat. Today Red Teaming, Crescendo, TAP, and PAIR each hand-roll the same mechanics: holding the adversarial system prompt, building the per-turn message, sending it on a stable conversation id, and parsing the reply. The manager owns one adversarial conversation and centralizes all of that in one place.

Because it's one place, it's also where we decide what context the adversarial chat gets each turn — the objective, the latest score, and the objective target's last response — all via a single adversarial_prompt_template. The manager computes the per-turn feedback text in Python (handling blocked / errored / empty responses and optional score feedback) and exposes it to the template as feedback_text, so adding a new signal becomes a template change rather than per-attack code.

It also unifies the shared adversarial_chat JSON schema and is the single home for JSON retry, so schema-aware targets natively constrain the reply and every attack gets consistent parsing/retry behavior for free.

Multimodal feedback

The manager folds in the modality router, so adversarial context is no longer text-only:

  • Seed media (e.g. images) provided on turn 1 is forwarded to the adversarial chat and preserved into the message sent to the objective target.
  • The objective target's latest response is bucketed by data type when composing feedback (e.g. {{ message.text.converted_value }}).
  • Image-generation adversarial system prompts (Crescendo / Red Teaming / TAP) now refer to provided seed images as editable inputs instead of describing them, so image targets edit/compose the seeds rather than regenerate them from scratch.

Config surface

AttackAdversarialConfig now carries system_prompt (inline string or SeedPrompt), first_message, and adversarial_prompt_template; the legacy system_prompt_path is deprecated (removed in 0.17.0). None of this is visible to callers of the executors — the public attack APIs are unchanged.

Migrated executors

Every consuming executor now routes its adversarial conversation through the manager: Crescendo, Red Teaming, Tree of Attacks (and PAIR), Simulated Conversation, Context Compliance, and Role Play. The scenario attack_technique_factory is updated to match.


OLD PR description by @rlundeen2

AdversarialConversationManager is a way to simplify managing adversarial conversations, letting us centrally give various kinds of context to the adversarial chat.

Today each multi-turn attack (Red Teaming, Crescendo, TAP, PAIR) hand-rolls the same mechanics: holding the adversarial system prompt, building the per-turn message, sending it on a stable conversation id, and parsing the reply. This manager owns one adversarial conversation and centralizes all of that in one place.

Because it's one place, it's also where we decide what context the adversarial chat gets each turn — the objective, the latest score, and the objective target's last response (bucketed by data type, e.g. {{ message.text.converted_value }}) — all via a single adversarial_prompt_template. Adding a new signal (a new score field, multimodal pieces, etc.) becomes a template change rather than per-attack code.

It also unifies the shared adversarial_chat JSON schema and is the single home for JSON retry, so schema-aware targets natively constrain the reply and every attack gets consistent parsing/retry behavior for free.

rlundeen2 and others added 6 commits June 17, 2026 17:04
Add a shared `adversarial_chat` JSON schema (next_message, rationale,
last_response_summary) and wire the Crescendo, TAP, and PAIR adversarial-chat
prompts onto it so their prompts are interchangeable. Parsers now read/return
next_message and forward the schema to schema-aware targets via prompt metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The message normalizer already appends the response JSON schema when it is
forwarded via prompt metadata, so the hand-written schema block in the
Crescendo system prompts duplicated that instruction.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Routes Crescendo and TAP adversarial sends through a real PromptNormalizer and
a MockPromptTarget (which lacks native JSON_SCHEMA support) to verify the shared
adversarial_chat schema is forwarded via prompt metadata and rendered into the
prompt the adversarial chat receives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eam-attack-schema

# Conflicts:
#	pyrit/datasets/executors/crescendo/crescendo_variant_1.yaml
#	pyrit/datasets/executors/crescendo/escalation_crisis.yaml
#	pyrit/datasets/executors/crescendo/therapist.yaml
#	pyrit/datasets/executors/pair/attacker_system_prompt.yaml
#	pyrit/datasets/executors/tree_of_attacks/adversarial_system_prompt.yaml
#	pyrit/datasets/executors/tree_of_attacks/image_generation.yaml

@romanlutz romanlutz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this! It solves lots of problems I've noticed but not really had a good handle on (yet!). That said, there are lots of conflicts with #1377. It's compatible if we adjust this or #1377. Some notes below.

  1. API migration: #2053 renames seed_promptfirst_message and adds adversarial_prompt_template. This would probably need deprecation etc.

  2. Responsibility boundary: #1377’s ModalityFeedbackRouter handles true multimodal forwarding (message pieces). #2053’s manager centralizes adversarial turn/schema logic but is text-message-centric in RedTeamingAttack. We'd need to generalize it to accept/build full Message objects (not just rendered text) so #1377 media forwarding remains intact.

  3. First-turn placeholder flow: #1377 supports adversarial placeholders + seed media for edit-only starts; #2053’s red-teaming path bypasses that behavior unless explicitly reintroduced.

  4. Rollout scope: apply manager only to red teaming first, or also refactor Crescendo/TAP to share the same adversarial-turn primitive.

Another thought that came to mind: Should there be an objective conversation manager then? It's worth considering, but as a follow-up if at all. There’s already ConversationManager; a new objective manager should only centralize objective-target send mechanics (rotation for single-turn targets, converter plumbing, execution context), not absorb attack-specific control flow (backtracking, tree branching, pruning).

So all in all: This PR is compatible with #1377 if integrated deliberately; a mechanical merge would likely regress #1377 multimodal behavior in red teaming. @rlundeen2

@rlundeen2

rlundeen2 commented Jun 28, 2026

Copy link
Copy Markdown
Contributor Author

Would you want to finish #1377 first? This PR is one I'd probably abandon and start over but I like the strategy. I'd be happy to pick up, but be even happier to offload 😁

Another strategy would be to tackle the goals of #1377 but using this class, which I am equally happy with

Copilot AI added 7 commits July 4, 2026 07:47
Resolve the collision between PR microsoft#2053 (AdversarialConversationManager)
and main microsoft#1377 (_ModalityFeedbackRouter) using Option A: the manager owns
the adversarial send/parse and, when a modality router is injected, builds
its outgoing message via router.build_adversarial_input_message so the
adversarial model still sees generated media (preserves microsoft#1377's multimodal
feedback loop). red_teaming keeps self._modality_router for the objective
side (placeholder fill / build_objective_input_message).

Preserve main's blocked/error/use_score_as_feedback semantics by baking
them into the default adversarial prompt template (DEFAULT_ADVERSARIAL_
PROMPT_TEMPLATE) rendered through _MessageView, rather than the removed
imperative _build_adversarial_prompt_async / _handle_adversarial_* helpers.

Note: the default red_teaming adversarial system prompt (text_generation.yaml)
declares no JSON schema, so the manager uses the raw-text path (adversarial
reply is the next message verbatim); the schema/JSON-parse path only engages
when a system prompt or first message declares a schema.

Test reconciliation:
- red_teaming: drop obsolete handler tests, rewire modality-router
  integration tests onto the manager/raw-text path, rename seed_prompt ->
  first_message.
- attack_config: add coverage for all six default-template branches
  (blocked / error / text / text+feedback / no-text+feedback / empty).
- simulated_conversation: cover schema vs raw-text next-message handling.
- shared attack tests updated for the first_message rename.

All 873 attack unit tests pass; ruff + ty clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the blocked/error/empty/score-feedback control flow out of the
per-turn Jinja template into a testable Python helper and render the
template strictly. Detect blocked/errored pieces across all message
pieces instead of only the first, fixing a bug where a blocked/errored
later piece was masked by an earlier clean one.

- adversarial_conversation_manager.py: replace the _MessageView /
  _MessageBucket template shim with _build_adversarial_feedback_text
  (+ _joined_text_value / _first_response_error); render via
  render_template_value (strict) exposing feedback_text + objective.
- attack_config.py: simplify DEFAULT_ADVERSARIAL_PROMPT_TEMPLATE to
  "{{ feedback_text }}".
- tests: relocate the template-branch coverage to
  TestBuildAdversarialFeedbackText (adding any-piece blocked/error
  cases) and update prompt assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… lifecycle

Address review findings microsoft#4-microsoft#6 on the adversarial conversation manager:

- microsoft#6 Schema-driven parsing (single source of truth): `_parse_adversarial_reply`
  derives its required/permitted keys from the resolved response schema (`required`,
  `properties`, `additionalProperties`) -- the same schema the manager forwards to
  constrain the target -- instead of a hard-coded copy of the adversarial_chat keys,
  so the parser cannot silently drift from adversarial_chat.yaml. Behavior is identical
  for the shared adversarial_chat schema; `next_message` is always required since the
  attack loop consumes it (enforced at parse time and, for the property, at construction).
- microsoft#5 Naming: rename the manager's `adversarial_first_prompt_template` param/attr/
  property to `first_message`, matching `AttackAdversarialConfig.first_message`.
- microsoft#4 Lifecycle: build the AdversarialConversationManager once per execution in
  RedTeamingAttack (`_build_adversarial_manager`, constructed before the turn loop
  and threaded into `_generate_next_prompt_async`) instead of rebuilding each turn.
- Drop the never-read `_adversarial_response_json_schema` field (keep the
  construction-time "declared on both" validation) and the dead `_DEFAULT_SEED_PROMPT`
  constant; refresh the stale `adversarial_prompt_template` comment.
- Thread the resolved schema through the simulated_conversation parse call.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Crescendo and TAP/PAIR each hand-rolled their own parser for the same adversarial_chat JSON schema. They now delegate to the shared _parse_adversarial_reply used by Red Teaming and Simulated Conversation, so validation and normalization stay consistent across every adversarial-chat executor.

This fixes TAP/PAIR's missing camelCase normalization (the exact gap that once broke a Crescendo CI run) and makes them enforce the schema's required keys and additionalProperties instead of only checking next_message. The schema argument is optional, so schemaless custom adversarial prompts keep the permissive next_message-only contract.

Also expands coverage: direct TAP node parser tests (camelCase accepted, strict-schema rejection of missing/extra keys, markdown stripping, schemaless-stays-lax), a manager next_message coercion test, a Red Teaming build-once test, and a Simulated Conversation invalid-JSON test. Removes the now-obsolete Crescendo _camel_to_snake test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rlundeen2's WIP renamed AttackAdversarialConfig.seed_prompt to first_message (field + factory internals) but left four tests in test_attack_technique_factory.py asserting the old field, causing AttributeError/TypeError. Update those tests to the new field name; the factory source was already migrated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…describe them

When seed/reference images are provided, the crescendo, red_teaming, and tree_of_attacks image_generation adversarial system prompts now instruct the adversarial model to refer to those images by position and issue an edit/compose instruction, rather than describing their contents. Describing the contents made image-editing targets regenerate from scratch and drop the seeded subject's identity. The seed guidance is conditional, so the no-seed generation path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Brings in 7 upstream commits (scenario DEFAULT strategy curation, dataset parameters on dataset configuration, initializer fixes, Azure SQL memory test fix, schema migration guard).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@romanlutz romanlutz changed the title Draft: AdversarialConversationManager Draft: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback Jul 8, 2026
@romanlutz romanlutz changed the title Draft: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback MAINT: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback Jul 8, 2026
@romanlutz romanlutz marked this pull request as ready for review July 8, 2026 16:59
@rlundeen2

rlundeen2 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Right now this only does RedTeamAttack. Which is good! But are there followups for the other attacks? Or we could do it here.

RN Crescendo, TAP, simulated_conversation all use some of it, but should (eventually) use the AdversarialConversationManager to take care of the next turn generally.

Comment thread pyrit/executor/attack/core/attack_config.py Outdated
Comment thread pyrit/executor/attack/multi_turn/red_teaming.py Outdated
Comment thread pyrit/executor/attack/multi_turn/red_teaming.py Outdated
Comment thread pyrit/executor/attack/component/adversarial_conversation_manager.py Outdated
Comment thread pyrit/executor/attack/component/adversarial_conversation_manager.py Outdated
Comment thread pyrit/executor/attack/multi_turn/red_teaming.py
Comment thread pyrit/executor/attack/multi_turn/red_teaming.py Outdated
Comment thread pyrit/executor/attack/component/adversarial_conversation_manager.py

@rlundeen2 rlundeen2 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good! The comments are scattered so I wanted a single "top-level" comment

AdversarialConversationManager should own the entire adversarial contract. You hand it the AttackAdversarialConfig, and it owns resolution of the schema, the system prompt, the first/next user messages, the modality placeholder weaving, and next-turn generation. Attacks should pre-resolve and set nothing — they just ask for the next message and get back a ready-to-send Message. Concretely:

  • Require schemas everywhere. Default to the canonical adversarial_chat schema, managed by the manager, overridable via config — and drop the raw-text path. Motivation: right now an inline string system prompt silently gets no schema and is never validated, and resolve_adversarial_json_schema returns None when neither prompt declares one, so "no schema" happens by accident rather than by choice. This means rewriting red_teaming's default text_generation.yaml to actually emit the JSON (schema + prompt are a matched pair) — I just did the same across all scorers and it was small and low-risk.
  • Manager owns setup. The set_system_prompt block and _set_adversarial_chat_first_message should move into the manager (it takes the config and does this). Just mind the two constraints: it must run before prepended-turn hydration, and it renders with max_turns.
  • Manager owns _generate_next_prompt_async, including modality placeholders. Attacks shouldn't branch on is_adversarial_placeholder. Folding this in also fixes a latent bug: the red_teaming bypass path returns the message without .duplicate() while Crescendo/TAP duplicate — three copies, already drifted.
  • Renames for clarity while they're still free (not released): adversarial_system_prompt / adversarial_first_user_message / adversarial_next_user_message.
  • Consider _AdversarialConversationManager — it's an internal component and I expect churn early on.

I'm open to either land it on RedTeaming here, and do Crescendo / TAP / SimulatedConversation as fast-follow, or doing all here.

Your objective-conversation-manager idea is worth a follow-up issue, but I agree it should only centralize objective send mechanics, not attack control flow.

Copilot AI added 8 commits July 8, 2026 15:19
…everywhere

The _AdversarialConversationManager now owns the entire adversarial-chat contract:
schema resolution (defaulting to the canonical adversarial_chat schema so every reply
is validated - no raw-text path), system-prompt setup, first/next user-message
rendering, per-turn send/parse/retry, modality placeholder weaving, and the bypass
path (now always .duplicate()s the seed, fixing the drifted red_teaming copy).

- Rename class to _AdversarialConversationManager; rename params to
  adversarial_system_prompt / adversarial_first_user_message / adversarial_next_user_message.
- Unify get_first_message_async + get_next_message_async into one entry point returning
  a ready-to-send AdversarialTurn; add override mode (adversarial_prompt_text) for
  Crescendo/TAP.
- Migrate RedTeamingAttack fully onto the manager; rewrite text_generation.yaml to emit
  JSON + declare response_json_schema_name: adversarial_chat.
- Delete AttackAdversarialConfig.get_json_schema(); warn (not raise) when a media-only
  objective response is dropped to a text-only adversarial.
- Reconcile manager/attack_config/red_teaming tests; add regression tests for
  bypass-duplicate, schema-everywhere fail-fast, empty-feedback warning, override mode.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both attacks now build their bespoke adversarial prompt text and hand it to the shared _AdversarialConversationManager, which owns schema resolution, send/parse/json-retry, the caller-seed bypass (always duplicated with fresh ids), placeholder filling, media forwarding, and objective-message construction.

Manager gains two capabilities the attacks need: set_adversarial_system_prompt now accepts **extra_render_values so attacks inject bespoke system-prompt inputs (Crescendo's conversation_context, TAP's desired_prefix/conversation_context) without rendering the prompt themselves; and a new generate_adversarial_reply_async send/parse entry point returns the parsed AdversarialReply without building an objective message, for TAP which scores next_message and may re-prompt before deciding what to send.

Removes Crescendo's _get_attack_prompt_async/_send_prompt_to_adversarial_chat_async/_parse_adversarial_response and TAP's _parse_red_teaming_response plus the per-node inline send/parse, eliminating three hand-rolled adversarial parsers so every adversarial reply is validated against the canonical adversarial_chat schema (camelCase-normalized, markdown-stripped, required-keys enforced). Adds regression tests covering the schema-everywhere contract and the new manager entry point.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Crescendo's _send_prompt_to_adversarial_chat_async and TAP's inline send were replaced by the shared manager, which parses every adversarial reply against the adversarial_chat schema. Drive Crescendo's assertion through _build_adversarial_manager(...).generate_adversarial_reply_async and back both attacks with a JSON-returning mock target so the real send/parse cycle completes while still asserting the schema reaches the adversarial target.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drives RedTeaming, Crescendo, and TAP end-to-end with a schema-aware adversarial reply that emits camelCase keys (nextMessage/lastResponseSummary) and asserts the normalized next_message reaches the objective target. This is the exact reply shape that once broke a Crescendo CI run under a hand-rolled parser; asserting it across every consuming executor guards against any of them regressing away from the shared schema-aware parser.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the is_adversarial_placeholder branch from the TreeOfAttacks node's
seed-handling path so the node no longer decides between bypass and
placeholder-fill itself. The seed message is handed to the manager's
get_next_message_async, which owns that decision, forwards seed media, and
fills placeholder slots. Drop the now-dead seed_message threading through the
red-teaming prompt helpers and split a merge-artifact test that had two bodies
concatenated. Add a bypass (no-placeholder) node test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…anager

Route the simulated-conversation final user message through the shared
_AdversarialConversationManager so schema resolution, system-prompt setting,
the adversarial send, JSON parsing, and pyrit_json_retry all live in one place.

- direct_next_message.yaml now declares response_json_schema_name: adversarial_chat
  and emits the matched JSON Response Format (schema + prompt are a pair), dropping
  the old raw-text output path.
- _generate_next_message_async delegates to the manager instead of calling
  set_system_prompt / send_prompt_async directly and branching on whether a schema
  was declared. It takes a prompt_normalizer the manager sends through.
- Tests exercise the manager path: JSON reply is parsed, a malformed reply is
  retried then raises InvalidJsonException (regression guard for the removed
  raw-text swallow), and a missing response raises a clear ValueError.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The AdversarialConversationManager now owns the full adversarial contract: a new resolve_config classmethod resolves the system prompt, coerces the first/next-message templates (applying canonical defaults in template mode), and fails fast on a duplicate response-schema declaration. Red Teaming (template mode), Crescendo, and TAP (override mode) call it once at construction and store the resolved bundle instead of coercing prompts themselves.

Removes the per-attack _set_adversarial_chat_first_message / _set_adversarial_prompt_template / _set_adversarial_chat_system_prompt_template helpers and prunes the now-unused imports, and adds resolve_config unit tests covering both modes, defaults, system-prompt resolution, schema-conflict fail-fast, and invalid-type handling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… TAP

RedTeaming already asserts that objective-response media is forwarded to a
capable adversarial chat (and dropped for a text-only adversarial chat), but
Crescendo and TAP thread last_response media through the same
_send_and_parse_async path without asserting it. Add matching
TestModalityRouterIntegration cases so all three manager-based attacks cover
the adversarial-input direction identically.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants